A clustering%2Dbased differential evolution for global optimization.

Experience%2C adjustment%2C and engagement the role of video in law enforcement.

Semi%2Dsupervised Probability Propagation on Instance%2DAttribute Graphs.

Learning Decision Trees with log Conditional Likelihood.

A Novel Bayes Model Hidden Naive Bayes.

Switching among Non%2DWeighting

Semi%2Dsupervised Self%2Dtraining for Sentence Subjectivity Classification.

Proper Model Selection with Significance Test.
Abstract. Model selection is an important and ubiquitous task in machine learning. To select models with the best future classification performance measured by a goal metric, anevaluation metric is often used to select the best classification model among the competing ones. A common practice is to use the same goal and evaluation metric. However, in several recent studies, it is claimed that using an evaluation metric (such as AUC) other than the goal metric (such as accuracy) results in better selection of the correct models. In this paper, we point out a flaw in the experimental design of those studies, and propose an improved method to test the claim. Our extensive experiments show convincingly that only the goal metric itself can most reliably select the correct classification models. 1
Discriminative parameter learning for Bayesian networks.
Bayesian network classifiers have been widely used for classification problems. Given a fixed Bayesian network structure, parameters learning can take two different approaches: generative and discriminative learning. While generative parameter learning is more efficient, discriminative parameter learning is more effective. In this paper, we propose a simple, efficient, and effective discriminative parameter learning method, called Discriminative Frequency Estimate (DFE), which learns parameters by discriminatively computing frequencies from data. Empirical studies show that the DFE algorithm integrates the advantages of both generative and discriminative learning: it performs as well as the state-of-the-art discriminative parameter learning method ELR in accuracy, but is significantly more efficient. 1.
Understanding Driving Activity Using Ensemble Methods.

Using Instance cloning to Improve Naive Bayes for Ranking.

Naive Bayes for optimal ranking.
Abstract. It is well-known that naive Bayes performs surprisingly well in classification, but its probability estimation is poor. In many applications, however, a ranking based on class probabilities is desired. For example, a ranking of customers in terms of the likelihood that they buy one’s products is useful in direct marketing. What is the general performance of naive Bayes in ranking? In this paper, we study it by both empirical experiments and theoretical analysis. Our experiments show that naive Bayes outperforms C4.4, the most state-of-the-art decisiontree algorithm for ranking. We study two example problems that have been used in analyzing the performance of naive Bayes in classification [3]. Surprisingly, naive Bayes performs perfectly on them in ranking, even though it does not in classification. Finally, we present and prove a sufficient condition for the optimality of naive Bayes in ranking. 1
Naive Bayes for optimal ranking.
Abstract. It is well-known that naive Bayes performs surprisingly well in classification, but its probability estimation is poor. In many applications, however, a ranking based on class probabilities is desired. For example, a ranking of customers in terms of the likelihood that they buy one’s products is useful in direct marketing. What is the general performance of naive Bayes in ranking? In this paper, we study it by both empirical experiments and theoretical analysis. Our experiments show that naive Bayes outperforms C4.4, the most state-of-the-art decisiontree algorithm for ranking. We study two example problems that have been used in analyzing the performance of naive Bayes in classification [3]. Surprisingly, naive Bayes performs perfectly on them in ranking, even though it does not in classification. Finally, we present and prove a sufficient condition for the optimality of naive Bayes in ranking. 1
A Switching Criterion for Intensification and Diversification in Local Search for SAT.
We propose a new switching criterion, namely the evenness or unevenness of the distribution of variable weights, and use this criterion to combine intensification and diversification in local search for SAT. We refer to the ways in which state-of-the-art local search algorithms adaptG 2 WSATP and V W select a variable to flip, as heuristic adaptG 2 WSATP and heuristic V W, respectively. To evaluate the effectiveness of this criterion, we apply it to heuristic adaptG 2 WSATP and heuristic V W, in which the former intensifies the search better than the latter, and the latter diversifies the search better than the former. The resulting local search algorithm, which switches between heuristic adaptG 2 WSATP and heuristic V W in every step according to this criterion, is called Hybrid. Our experimental results show that, on a broad range of SAT instances presented in this paper, Hybrid inherits the strengths of adaptG 2 WSATP and V W, and exhibits generally better performance than adaptG 2 WSATP and V W. In addition, Hybrid compares favorably with state-of-the-art local search algorithm R+adaptNovelty+ on these instances. Furthermore, without any manual tuning parameters, Hybrid solves each of these instances in a reasonable time, while adaptG 2 WSATP, V W, and R+adaptNovelty+ have difficulty on some of these instances.
Probability Based Metrics for Locally Weighted Naive Bayes.

Learning Locally Weighted C4.4 for Class Probability Estimation.

A Context Aware Automatic Traffic Notification System for Cell Phones.

Naturalistic use of cell phones in driving and context%2Dbased user assistance.

Combining Adaptive Noise and Look%2DAhead in Local Search for SAT.
Abstract. The adaptive noise mechanism was introduced in Novelty+ to automatically adapt noise settings during the search [4]. The local search algorithm G 2 W SAT deterministically exploits promising decreasing variables to reduce randomness and consequently the dependence on noise parameters. In this paper, we first integrate the adaptive noise mechanism in G 2 W SAT to obtain an algorithm adaptG 2 W SAT, whose performance suggests that the deterministic exploitation of promising decreasing variables cooperates well with this mechanism. Then, we propose an approach that uses look-ahead for promising decreasing variables to further reinforce this cooperation. We implement this approach in adaptG 2 W SAT, resulting in a new local search algorithm called adaptG 2 W SATP. Without any manual noise or other parameter tuning, adaptG 2 W SATP shows generally good performance, compared with G 2 W SAT with approximately optimal static noise settings, or is sometimes even better than G 2 W SAT. In addition, adaptG 2 W SATP is favorably compared with state-of-the-art local search algorithms such as R+adaptNovelty+ and V W. 1
A Fast Decision Tree Learning Algorithm.
The choice of an input representation for a neural network can have a profound impact on its accuracy in classifying novel instances. However, neural networks are typically computationally expensive to train, making it difficult to test large numbers of alternative representations. This paper introduces fast quality measures for neural network representations, allowing one to quickly and accurately estimate which of a collection of possible representations for a problem is the best. We show that our measures for ranking representations are more accurate than a previously published measure, based on experiments with three difficult, real-world pattern recognition problems. 1 Introduction  A key component of successful artificial neural network (ANN) applications is an input representation that suits the problem. However, ANNs are usually costly to train, preventing one from trying many different representations. In this paper, we address this problem by introducing and evaluating three ...
Learning Naive Bayes for Probability Estimation by Feature Selection.

Lazy Averaged One%2DDependence Estimators.

Probabilistic Inference Trees for Classification and Ranking.

Improving the Ranking Performance of Decision Trees.
Archives des publications du CNRC (NPArC)
Dynamic K%2DNearest%2DNeighbor Naive Bayes with Attribute Weighted.

Full Bayesian network classifiers.
The structure of a Bayesian network (BN) encodes variable independence. Learning the structure of a BN, however, is typically of high computational complexity. In this paper, we explore and represent variable independence in learning conditional probability tables (CPTs), instead of in learning structure. A full Bayesian network is used as the structure and a decision tree is learned for each CPT. The resulting model is called full Bayesian network classifiers (FBCs). In learning an F BC, learning the decision trees for CPTs captures essentially both variable independence and context-specific independence. We present a novel, efficient decision tree learning, which is also effective in the context of F BC learning. In our experiments, the F BC learning algorithm demonstrates better performance in both classification and ranking compared with other stateof-the-art learning algorithms. In addition, its reduced effort on structure learning makes its time complexity quite low as well. 1.
Decision Trees for Probability Estimation An Empirical Study.
Archives des publications du CNRC (NPArC)
Weightily Averaged One%2DDependence Estimators.
This paper describes a method based on data mining techniques to classify MIDI music files into music genres. Our method relies on extracting high level symbolic features from MIDI files. We explore the effect of combining several data mining preprocessing stages to reduce data processing complexity and classification execution time. Additionally, we employ a variety of probabilistic classifiers and ensembles. We compare the results produced by our best classifier with those obtained by more complex state of the art classifiers. Our experimental results indicate that our system constructed with the best performing combination of data mining preprocessing components together with a Naive Bayes-based classifier is capable of outperforming other more complex ensembles of classifiers. 1
Identification of Real%2DTime Diagnostic Measures of Visual Distraction With an Automatic Eye%2DTracking System.

Learning probabilistic decision trees for AUC.

Representing Conditional Independence Using Decision Trees.

Hidden Naive Bayes.
Naive Bayes is an effective and efficient learning algorithm in classification. In many applications, however, an accurate ranking of instances based on the class probability is more desirable. Unfortunately, naive Bayes has been found to produce poor probability estimates. Numerous techniques have been proposed to extend naive Bayes for better classification accuracy, of which selective Bayesian classifiers (SBC) (Langley &amp; Sage,
Learning k%2DNearest Neighbor Naive Bayes for Ranking.

One Dependence Augmented Naive Bayes.
Abstract. In real-world data mining applications, an accurate ranking is same important to a accurate classification. Naive Bayes (simply NB) has been widely used in data mining as a simple and effective classification and ranking algorithm. Since its conditional independence assumption is rarely true, numerous algorithms have been proposed to improve Naive Bayes, for example, SBC[1] and TAN[2]. Indeed, the experimental results show that SBC and TAN achieve a significant improvement in term of classification accuracy. However, unfortunately, our experiments also show that SBC and TAN perform even worse than naive Bayes in ranking measured by AUC[3, 4](the area under the Receiver Operating Characteristics curve). This fact raises the question of whether can we improve Naive Bayes with both accurate classification and ranking? In this paper, responding to this question, we present a new learning algorithm called One Dependence Augmented Naive Bayes (simply ODANB). Our motivation is to develop a new algorithm to improve Naive Bayes ’ performance not only on classification measured by accuracy but also on ranking measured by AUC. We experimentally tested our algorithm, using the whole 36 UCI datasets recommended by Weka[5], and compared it to NB, SBC[1] and TAN[2]. The experimental results show that our algorithm outperforms all the other algorithms significantly in yielding accurate ranking, yet at the same time outperforms all the other algorithms slightly in terms of classification accuracy. 1
Driver State Monitor from DELPHI.

Instance Cloning Local Naive Bayes.

Learning Tree Augmented Naive Bayes for Ranking.

Learning Instance Greedily Cloning Naive Bayes for Ranking.

Augmenting naive Bayes for ranking.
Naive Bayes is an effective and efficient learning algorithm in classification. In many applications, however, an accurate ranking of instances based on the class probability is more desirable. Unfortunately, naive Bayes has been found to produce poor probability estimates. Numerous techniques have been proposed to extend naive Bayes for better classification accuracy, of which selective Bayesian classifiers (SBC) (Langley &amp; Sage,
Exploring Conditions For The Optimality Of Na%EFve Bayes.

Naive Bayesian Classifiers for Ranking.
Abstract. It is well-known that naive Bayes performs surprisingly well in classification, but its probability estimation is poor. In many applications, however, a ranking based on class probabilities is desired. For example, a ranking of customers in terms of the likelihood that they buy one’s products is useful in direct marketing. What is the general performance of naive Bayes in ranking? In this paper, we study it by both empirical experiments and theoretical analysis. Our experiments show that naive Bayes outperforms C4.4, the most state-of-the-art decisiontree algorithm for ranking. We study two example problems that have been used in analyzing the performance of naive Bayes in classification [3]. Surprisingly, naive Bayes performs perfectly on them in ranking, even though it does not in classification. Finally, we present and prove a sufficient condition for the optimality of naive Bayes in ranking. 1
Conditional Independence Trees.
Abstract. It has been observed that traditional decision trees produce poor probability estimates. In many applications, however, a probability estimation tree (PET) with accurate probability estimates is desirable. Some researchers ascribe the poor probability estimates of decision trees to the decision tree learning algorithms. To our observation, however, the representation also plays an important role. Indeed, the representation of decision trees is fully expressive theoretically, but it is often impractical to learn such a representation with accurate probability estimates from limited training data. In this paper, we extend decision trees to represent a joint distribution and conditional independence, called conditional independence trees (CITrees), which is a more suitable model for PETs. We propose a novel algorithm for learning CITrees, and our experiments show that the CITree algorithm outperforms C4.5 and naive Bayes significantly in classification accuracy. 1
The Optimality of Naive Bayes.

Learning Conditional Independence Tree for Ranking.
Accurate ranking is desired in many real-world data mining applications. Traditional learning algorithms, however, aim only at high classification accuracy. It has been observed that both traditional decision trees and naive Bayes produce good classification accuracy but poor probability estimates. In this paper, we use a new model, conditional independence tree (CITree), which is a combination of decision tree and naive Bayes and more suitable for ranking and more learnable in practice. We propose a novel algorithm for learning CITree for ranking, and the experiments show that the CITree algorithm outperforms the stateof-the-art decision tree learning algorithm C4.4 and naive Bayes significantly in yielding accurate rankings. Our work provides an effective data mining algorithm for applications in which an accurate ranking is required. 1
Learning Weighted Naive Bayes with Accurate Ranking.
Naive Bayes is one of most effective classification algorithms. In many applications, however, a ranking of examples are more desirable than just classification. How to extend naive Bayes to improve its ranking performance is an interesting and useful question in practice. Weighted naive Bayes is an extension of naive Bayes, in which attributes have different weights. This paper investigates how to learn a weighted naive Bayes with accurate ranking from data, or more precisely, how to learn the weights of a weighted naive Bayes to produce accurate ranking. We explore various methods: the gain ratio method, the hill climbing method, and the Markov Chain Monte Carlo method, the hill climbing method combined with the gain ratio method, and the Markov Chain Monte Carlo method combined with the gain ratio method. Our experiments show that a weighted naive Bayes trained to produce accurate ranking outperforms naive Bayes. 1
A Fundamental Issue of Naive Bayes.
Naive Bayes is one of the most efficient and effective inductive learning algorithms for machine learning and data mining. But the conditional independence assumption on which it is based, is rarely true in real-world applications. Researchers extended naive Bayes to represent dependence explicitly, and proposed related learning algorithms based on dependence. In this paper, we argue that, from the classification point of view, dependence distribution plays a crucial role, rather than dependence. We propose a novel explanation on the superb classification performance of naive Bayes. To verify our idea, we design and conduct experiments by extending the Chow-Liu algorithm to use the dependence distribution to construct TAN, instead of using mutual information that only reflects the dependencies among attributes. The empirical results provide evidences to support our new explanation.
AUC A Better Measure than Accuracy in Comparing Learning Algorithms.
Predictive accuracy has been widely used as the main criterion  for comparing the predictive ability of classification systems (such  as C4.5, neural networks, and Naive Bayes). Most of these classifiers also  produce probability estimations of the classification, but they are completely  ignored in the accuracy measure. This is often taken for granted  because both training and testing sets only provide class labels. In this  paper we establish rigourously that, even in this setting, the area under  the ROC (Receiver Operating Characteristics) curve, or simply AUC,  provides a better measure than accuracy. Our result is quite significant  for three reasons. First, we establish, for the first time, rigourous  criteria for comparing evaluation measures for learning algorithms. Second,  it suggests that AUC should replace accuracy when measuring and  comparing classification systems. Third, our result also prompts us to  re-evaluate many well-established conclusions based on accuracy in machine  learning. For example, it is well accepted in the machine learning  community that, in terms of predictive accuracy, Naive Bayes and decision  trees are very similar. Using AUC, however, we show experimentally  that Naive Bayes is significantly better than the decision-tree learning  algorithms.
A Fundamental Issue of Naive Bayes.
Naive Bayes is one of the most efficient and effective inductive learning algorithms for machine learning and data mining. But the conditional independence assumption on which it is based, is rarely true in real-world applications. Researchers extended naive Bayes to represent dependence explicitly, and proposed related learning algorithms based on dependence. In this paper, we argue that, from the classification point of view, dependence distribution plays a crucial role, rather than dependence. We propose a novel explanation on the superb classification performance of naive Bayes. To verify our idea, we design and conduct experiments by extending the Chow-Liu algorithm to use the dependence distribution to construct TAN, instead of using mutual information that only reflects the dependencies among attributes. The empirical results provide evidences to support our new explanation.
AUC a Statistically Consistent and more Discriminating Measure than Accuracy.
Predictive accuracy has been used as the main and often only evaluation criterion for the predictive performance of classification learning algorithms. In recent years, the area under the ROC (Receiver Operating Characteristics) curve, or simply AUC, has been proposed as an alternative single-number measure for evaluating learning algorithms. In this paper, we prove that AUC is a better measure than accuracy. More specifically, we present rigourous definitions on consistency and discriminancy in comparing two evaluation measures for learning algorithms. We then present empirical evaluations and a formal proof to establish that AUC is indeed statistically consistent and more discriminating than accuracy. Our result is quite significant since we formally prove that, for the first time, AUC is a better measure than accuracy in the evaluation of learning algorithms. 1
Numeric Mapping and Learnability of Naive Bayes.

